Automatic Evaluation of Wordnet Synonyms and Hypernyms

In recent times, wordnets have become indispensable resources for Natural Language Processing. However, the creation of wordnets is a time consuming and manpower intensive proposition. This fact has led to attempts at quickly fixing a wordnet using text repositories such as the web and certain corpora, and also by translating an existing wordnet into another language. However, the results of such attempts are often far from ideal, in the sense that the wordnet so produced contains synsets that have outlier words and/or missing words. Additionally, semantic relations may be inappropriately set up or may be missing altogether. This has necessitated investigations into automatic methodologies of wordnet evaluation. This is very much in line with modern NLP’s insistence on concrete evaluation methodologies. To the best of our knowledge, the work reported here is the first attempt at an automatic method of wordnet evaluations. We focus on verifying synonymy within non-singleton synsets and also on hypernymy between synsets. Assuming the Princeton WordNet to be the gold standard, our method is shown to validate 70% of all non-singleton synsets and about the same proportion of hypernymy-hyponymy pairs.